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Final Evaluation Report 
1973-74 ^ 

EXECUTIVE SUMMARY 



During the past year ( 1973-74) the Anchorage Borough School District 
designed and conducted^ a Title I program. - The p-tograra f ocusedN^n ♦ the im- 
provement of reading and language skills among educationally di^advan-tased - 
ch*ildren within four of the District's elementary schools. Specific ob- _ 
jeotives of 'the progratn included enhanced reading skills, enhanced language 
skills, and nastery of bSsic concepts in the areas of food "handling, com- 

K ' . ' 

munications, transportation and career awareness. 

• The instructional program wljich was designed to assipt children in 
meeting these objettives consisted of four activities: (a) the Southwest 
Cooperative Educational Laboratory Oral Language Program, (b) a locally <• 
d^fesigned read ing ^ss istance . program , (c) a locally designed language 
development program, am} (d) a Home-School Coordination program. In all,, 
some 53 children at the kindergarten level, were served, and 75, 68, and' ' • 
73 children were served at the .first, second, and^third grade level.s, 
respectively for tne entire year. ^ 

During the year, two major work areas (or components) supported the 
.program's instructional activities. Within, the program's Mahag.eraeht Com- 
ponent, sixteen objectives yere established. '.These objectives included 
such functions as development and monitoring of the program, provision*^ of 
training for the staff, preparation and submission of the requisite . reports 
All sixteen of the Management objectives were attained by the end of the 
proj ect year . ' , 



V.'ithin thve prd-^ran's Staff nnd Materials Deveiopment Cccponent, a * - . 

total "cf five Staf-f Dev^Iopnent 'Objectives and fiye Materials Developnenc 

objective^ were established. The Staff Development obj e^ctives > focused 

primarily upon preservice and inservice activities as »well as thfe appro- 

priate enactment of^ procedures and skills within the p2::^gran classrooms. 

Three of five objectives within this area \^ere clearly attained, 
i » , 

Work outll/ned by the Materials Deveiopment objectives centered about 
the production of instructional lessp;^ -plans for use in the basic concept . 
areas as well as tha devlloptnent and production of program assessment in- 
struments. Two o.f the objectives within this group were clear.ly attained 
by the end of the year'. ^ 

^. In reviewing the performance of participating children at least- 
three inajor findings w^re observed: 

• The percentages of children who attained mastery level in the 
* program*'s concept areas (i.e.^ transportation, food marketing, 
communication and career awareness) were not high. A , recopnnendation 
was made that, a hierarchy of concepts suitable to children in the 
♦ grade level^served ,be- established. 



Student performance on the Southwestern Cooperative Educational 
Labotatory (SWCEL) Oral Language Test was generally favorable-. 
It was conluded that both project children and control children . 
gained substaht^alLy in their language performance, during the 
year, but that the growth of the -project children was considerably 
greater than that, of- the children who did not receive SWCEL 
instruction. 

Student performances on a st>an,dardized reading test were compared 
with those of students who had been enrolled in the District's 
Title I program during 1971772 and 1972-73. For the most^part, 
present project ' students .averaged between eight and ten months' 
growth in _r6adihg during the year, but th-ey generally remained, 
behind the expected or average grade level in performance. A 
greater concentration of .effort in the reading area was 
recommended. * , , * ' ' 



ii 



The program staff provided a number of insights into project oper- 
ations (see pages 60-61 of the present report) and a number of program 
produc ca "were identified (see page 59). r 



Recornnenda tions were made- by the Program Evaluator in the areas of 
Ca) internal program monitoring and communications (b)^ teacher ratings 
^^f Student performance, (c) test scheduling (d) process evaluation pro- 



"cedures, (e) project staffing and (f) instructional procedjires. 



\ 
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INTRODUCTION TO THE PROJECT 



A . Project Rationale 

^ Within the Dist;rict ' Title ' I schools, between 20 and 30 percent^ of 

the students (grades K-3) have identifiable leaini'iiig problems related to 

oral language facility and reading skills. In the past, failure to 

provide these young^t^rs with an appropriate* experiential background either 

at hone or in the school has resulted in a continul^ lag in learning 

progress in later grades. ' The - purpose ^of the present- program is to ... 

' ' > ' «' 

provide experiences through 'whiqh atuderrts may. develop sdme essential 

r, " ' « 

' ^ ( , ' . A ' . ■ . - 

basic concepts. , • ^ 

It was anticipated that. once in possession of:a s£t of core toncepts 
which are common to a^l students in a given classroom^, ^the disadvantaged 
student would be in a better position to assimilate ' classropm instructio^. 
As a result of this, the student's acquisition of oral and written 
language skills' should be maximized. ^ 

B. Project Focus ' . » • . . 

During. the 1973-74 school year, the District'^ Title I program 
provided field trip e^^periences to Project participants (grades K-1) in 
the areas of t ranspprtation , communication^ marketing and career 
infoniation. The direct ^cla^srocr'm extension of th^se experiences results 



in*^he development of ^ the core <:oncepts. Additionally, oral language 
specialists ar.d reading specialists provi'ded services to each Title I 
classroom. ' Another significant feature of the program wa^ the use of a 
Hone-Scbool Coordinator. This individual (one at each of the Project 
' schools) w as responsi6le for establisjiing and maintaining liaison between 
the teaching staff and the child's family. Finally, the S^outhwest'^ 
Cooperative Educational Laboratory Oral Language Progr^am (OLP) was 
provided to selected Title I children.. ' « ♦ . 



II 

DES^CRIPTION OF TARGET POPIJLATION . 

i 



The present Project was designed to serve selected students in grades 
K-3 in foqr elementary schools within ' the^nchorage Borough School District. 
Selectioq^of the target schools was biased upon an economic criteria, .i.e., 

A 

those schools within the district having the highest caricentration of AFDC 
children were^ designated as Title 1 schools. By- means of this criteria, 
four schools (Jtt^nali^ North Star, Mountain ^View and Fairvlew) were selected 
from among the district's 41 elementary schools. 

WithJLn each,o-f the target schools, educationally disadvantaged children 

V • 

in* grades K-3 were identified by means of: 

1. Performance on standardized tes'ts* and/o-r 

* 

2. Teacher or curricular specialist r ecoininendation 

In formulating such recommendatiQns , teachers and curricular specialists 
made judgments- during the fall t-er.m about e^ch child enrolled in* classes 
(K-3) within the target school. The judgments focused on six areas: 

♦ 1. Conceptual background 

2 . Extent of vocabulary ^ ^ 

3. Syntax / * , 

4. Social Adjustment *^ jt) 

5. Self-concept ; ' ^ . 

6. Bilingual^r^ dual-cultural background 



En the present instance, children were considered educdkcionally disadvaatag 
if: fa) they scored at or below the C level on* the Metropolitan Readiness 
Test (^rade one only), or (b) scored below grade level -(.5 grade level or 
more) on the Stanford Achievement Test'-. 



For Che benefit , of children entering t}\e schools after th^ fall 
•asjiessmen-ts , individual ratings were^ made by teachers, and specialists 



on 



an "as nec^ded" t^asis. Thus, of the 1,081 children ^nrol led in grades 



K-3 in the target schools during the fall, approximately 77 were selected 

intotVie Title 1 Program in Kindergarten. In 'addition, 118 were selected 

from tfne firSc^gr^de^ 1£)5 were selected from thg' second grade and 103 
r 

were ^elecceci. fi^om the third grade. ^ 

D\rin^. the course of the/year, the Project experienced considerable ^ 
turnov©^ in students. Thus, an additional 20 students were subsequently 
served in Kindergarten, 34 were serv^ed in first grade, 33 were served in^ 
second grade ^nd 33 were served ih third grade. a result of turnover, . 

27. df ^bhe. b'riginally selected Rirylergarten students, 38 of the first 
grade stud'erTts, 39^pf ttf^ second grade students and 37 of the third 
grade students- left the program.'' Finally, one kindergarten ten ffrst grade, 
six second grade and six third gr^de students left and then . re j oined the . 
program on.at. least one occasion. In all, some 5-3 Kindergarten students, 
75 first grade *sJtudAtsv, 68 secotvd grade students and 73 third grade students 
remained with the program for the entire year. It is on the basis of 
this latter group of ''pure cases" that the effectiveness of the progi^an is 
J to be adjudged. ^ ' \ ' 



■ • , .III 

PROJECT OPERATIONS 

^ . In ch^s section, major aspects of the Project operation are detailed. 
Specifically, the topics of: (a) the Proje.ct Model, (b) the Project 
Organization, (c) the^Pcoject Evaluatiofu, (d) the Staffing Summary, (e) tlje 
Trailing. of Prpject'Staff, and (f)^ the Chronology of Major Events are 
presented. ' v • 

A. . Project M^d-el 

The Project's instructional procedures cluster about foufr major 

activities. In the paragraphs which follow, each of these activities 

is described briefly.^ An overview of the program offerings at each 

"of the Project schools is provided in Figure 1, Page lO. 

». « 

•i - ^ 

1. SOUTHWEST COOPERATIVE EDUCATIONAL LABORATORY ORAL LANGUAGE PROGRAM 
(SWCEL-OLP)- ' • 

The SWCEL .Oral Language Program focuses on the production of 
English language. The instruction^\guide, Mark 111 Oral Language-^ 
Program, SWCELtil971, consists of si/voj^umes and provides^he basis 
for the program. Each volanfe consists of approximately 25 lessons. 
Each pf the 15D lessons contains: (a) explicit objectives, 
(b) specific methods of presentation, and (c) a listing of instruc- 
t Lonal. materijils . In additron to the guides, seven program tests • 
and most >of the necessary materials for the lessons are included as 
a -package. ' 

19 
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In operating the program, children are initially selected ' 
ori the basis'of performance on'the SWCEL Test of Oral^.English 
Prcrduction. For those children who have been selected^, daily 
lessons (usually of 20 to 25 minutes, duration) are provided. The - 
lessons are usually presented by the teacher or aide in a small 
group -Betting. Periodic tests of progiress are administered approxi- 
mate?cy every six weeks. 

The furtctibn'of the QUality Assurance ^Specialist (QAS) is. 
also an important aspect of the instructional procedure. At least 
monthly, a^ Quality Assurance Specialist reviews the instructional 
techniques of the t^eacher or aide. Feedback to the teacher or 
aide is usually provided immediately following the observation. 

M STRICT ORAL .LANGUAGE ..PROGRAM \ . ^ . ' 

" ^ The District? Oral Language Program focuses on developing 
core concepts in the areas, of Food Marketing, Communication, 
Transportation and Career Awareness. To that end, the District's 
Oral Language* SpecJ.alists provide lessons and coordinate field 
trips^for participating students. During the past year some 41 
separate lesson plans were developed and enacted bx, Ora^an'guag^ 
Specialist|s and Title I teachers. ' As in the case of the S'UCEL OLP, 
the l|sson plans adhere to a common format which includes: 
(a) specification of general and specific concepts, (b) listing 
and location of supporting materials, (c) description of instruc- 
tio^al acClvitids and field trips, and (d) suggested test items ^ 
for the concept being taught. . ^ 
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Duriq^ the'" past year , the District Oral Language P^sj^ara 
was offered to selected Title I children. Selection was based 
upon the judgment of- Title I staff members. 



TITLE I READING PROGRAM ■ 

The Title l' Reading ^Program consists of su{)plementary reading 
instruction which is provided by a specialist in eath building. 
The Reading Specialist works either with individuals or small 
groups of children on an "as needed" basis. Thus, the services 
of the Reading Specialist are provided to Project ^children for 
varying amounts of time within a given day and for varying lengths 

.of time during the yiear . In helping the Project children, the ^ 

♦ ■■ 

specialists work in a separate room in each building. . The rooms 

are equipped with a broad range of instructional material? and 

equipment. During th,e/past year, the specialists worked primarily 

with Project students in grades two and three. 

p 

H^ME-SCHOOL COORDINATION SERVICES ' * ^ 

If ^ 

The Home-School Coordinators provide a liaison between the 
home, the school and the community. Much of the work o'f the Home- 
School Coordinator entails: (a) working with individual children, 
(b) visiting the parents in their homes, (c) sharingx information 
between home and school, and (d) establishing and coordinating 
local parenf advisory groups. 

Project children at all grade, levels were served by the 

Home-school Coordinators during the past year. 

\ 
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B'. Pro i ect Organization 

This section of the, report includes an outline of all Project, 

^ objiifcLves organized according to component. The outline serves to 

orient the reader to the Project's organization, as well as to 

identify all Project objectives^ in an Abbreviated form. This 

outline is presented in Table I'below: 



\ 



TABLE 1 

Summary of Project Objectives 
Organized According to Component 

MANAGEMENT COMPONENT 

Objectives and dates of activation/completion 
a. 



Project Director forms program 
advisory councils 

Advisory groups prioritize 
studeot needs 

^Project staff. Project 
Director identify target 
populations and locations 

Project staff specifies 
student performance 
obj^iectives 

Project staff /Project 
Evaluator develop pre- 
liminary Evaluation Plan 

PiToject staff develops \ 
program proposal 

Project Director submits 
proposal for funding 



August-Septetiber , 1973 
January, 1973 

March, 1973 

March, 1973 

April, 1973 * 
April, 1973 

May, 1973 



ERLC 
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h. Project' operations formally 
activated by Project Director 

i. Project Director arranges for 

Project sites, supplies, peTr- 

sonnel and consultants 
* 

j. Project Director submits 
(Quarterly repoirts 



July,- 1973 



July, 1973 



, October, February, 
April and July ^ 
■ (annu^ly) 



k. Final Evaluation Plan ^ 
f ormalizjed by^ consultant 
and sta^f 

1. Project Director provides 
. preservice ^nd" inservi.ce 
training * ' 

m. Project Director (or his 

designate) monitors program 
implementatioV at the school 
and District/ level cfn a 
monthly basis " 

n. Project Evalu^tor and staff , ^ 
implement Project Evaluation 
Plan 

i 

] o. Parents are informed of^ 
program by Project staff 

*p. Home^School Coordinator 

establishes and maintains 
contact with parents of Title 
I children 

II • STAFF AND MATERIALS DEVELOPMENT COMPONENT 



July, 1^73 



July, and as needed 



September-May 

June -July 
< 

September-May 



September-May 



A. Staff l)evelopmentM)bj;ectives 

1. Project staff details training obje<:tives and 
activities for general Project procedures 

2. Project consultants detail training objectives 
• and procedures for prepackaged instructional 

program v * 

* New objective January, 1974 , 
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3, Project' staff attends trainiag sessions 

4, Project staff a'ftain training objectives 

• . ' • * • . , ^ 

5, Classroon monitoring by building aftd Distri^ct 
supervisory personnel serves as the basis for 
(supplementary) inservice sessions 



B, Matej-ial Deyeldprnen-t Ob^ctives 



1, Project staff r^\{iew District inst^rxictional 
materials ^ ^ ' * 

2, Project JDirecCbr . vrites material/concept 
specifications ' . 

3, * Project st'aff, develop, adapt or adopt materials/ 
concepts aocordlng to specif icatdons ' 

- • # ■ * 

4, Project Evaluator defines needs for program 
assessment devices 

^5, Project Evaluator constructs (or supervisees 

construction of) necessary assessment devices 

li^, - INSTRUCTIONAL COMPONENT ,^ 

A, Mastery o^f transportation, communication, food marketing 
concepts obj ectives 

(Product) 1/ Project students (grades K-3) x demonstrate 
< *^ lAastery of major concepts in, areas of 

transportation , communication , food marketing 

(Process) 2. Project staff provides fie^d trips in ;foregoing 
areas 

B, Mastery of career information concepts objectives 

(Product) 1. Project students (grades K-3) demonstrate 

, mastery of major concepts in career information 
f areas 

(Process) 2, Project staff provides field trips 

C, >Iastery of oral language skills objectives 

(Pi;oduct) 1. project students (grades K-3)- demonstrate 
mastery (gains in proficiency)' or oral 
language, skills 

(Process) 2. Project staff provide field trips and formalized 
oral language cur,riculum to students 



D- Mastery of reading skills objectives 

(Product) 1. Project students (grades 1-3) <i.®monstrate 

satisfactory level of attainment in reading 
skills 
V " 

(Process) 2, Project staff provide, field trips , formalized' 
oral language curriculum and locally developed/ 
, defined reading curriculum \ 
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C> Project Evaluatio n 

• In general^ the P^^oject's evaluation design •foe us.ed upon the 

\ • • • ' 

assessnent of ipdividual Project objectives. Thus, at leaqt one 

evaluative question was detailed for each objective. The procedure 

for answering each of the specified evaluative questions was then 

incorporated into the Project Evaluation Plan (NWREL, July 1972). / 

It should be -noted that Project evaluation was accopiplished 

by means of a contractual agreement with* a third party (Audit and 

^valuation Section, Northwest Regional Educational Laboratory) . 

The Project 'Evaluation Plan was jointly developed by the Project staff 

t 

ahd members of the? NWREL Evaluation Team. Project staff members were 

■ ■■ . • '^"^ . • . ' ■ ■ ■ 

responsible for the implementation of the data collection and the data 
reduction procedures outlined by the Evaluation Plan. The data analysis 
and reporting function* were carried out by the Evaluation Team members. ^ 
In addition to the present teport, three briief accomplishment reports w^re 
' produced in behalf/ of. the , Project by the Evaluation Team in October, 1973, 
January 1974 and 'April 1974, respectively. These reports considered the 
.status of objectives scheduled for implementation at the time of each 
report . The summary of the preceding reports is contained uttder the 
section "Pro'ject Outcomes." 
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j)^ Staffing Summary ^ 

During the present 7ear, funds from the Anchorage 'BorougTi Title i 
program were used to obtain the ^services of. a variety of individuals •> 
C^ee Figure 1, Pagei^ . .^Specifically, program' funds provided 
for: _ 

1. " central office staff '(.75 FIE) . 

V 

2. specialists and aides within the program schools and 

3. technical assistance and consultation. 

Principal duties of 'each of the Title I s^aff membe^rs and consultants 

are de^scribed below: - * * • ^• 

' .-^ - • • ' 

The Anchorage Borough School District provided facilities and sta^f 
to the Project, and also served 'as fiscal agent to the program. 

- - . ■ ■ ■ / 

The Director of the Divisian of Program Development , Dr. William Marsh 
was responsible for the needs assessment study, which served as a. basis 
for the Projects Dr^ Marsh was also instrumental in forming the Project 
Advisory ^Council, in preparing the program proposaJ.s, in supervising / 
Project operations, in selecting staff, and in selecting and supervising 
contracted technical assistance . 

The Title I Coordinator, Mrs. Gayle De- Sautel , was responsifel^ for: 

1. Monitoring the program on a weekly basis. ^ ' 

2. Coordinating material^ development and staff training. 

3. Presiding at Central Parent Council meetings. ""^ 

4. Implementation of local aspects of program Evaluation Plan. 

5. Implementation of the prbgram budget. 

6. Preparation of narrative reports required by the SDE. 

7. Maintaining liaison with the program technical assistance 
contractors. 

♦ 



T he Central ^Parent Council , consisted of the eight members . The^ 

Council was responsible for roviewin?; and recommending Title I programs 

for the entire Distritt. Th-e Council met monthly to review program 

polici-es, procedures and activitied. During Che past year, the following 

irpdividuals served on the Council on -a regular basis: 

Mrs. Sheila Howe 
Mr; Earl Mansur 
'Mrs. .Rosemary Rodriquez 
Mfs . Cora Tootkaylok .» 
Mr, Cecil Griffin 
Mrs. Janice fcCuphalt ' 

Each of the Council members had one or more children In the District's 

Title^ I schools. ' ^ ■ ^ " 

The Responsibilities oi the Title I Building Pirincipals included the 
following: 

• l/iring Project staff members within their Respective- buildings. 

• Implementing the- general program within each of the four buildings. 

Init iat ing "^expenditures of Title I program funds and arranging for 

• the release of staff members when training or materials development 
activities occurred. 

t 

Meeting iwith the Project Management team for purposes of program 
coord inat 

• • Scheduling staff meetings' within each building when necessary. 
The following indiviiuals served as Title, I Building Principals 
during the past year: 

Kennet'^ Baker 
Willis^Williams 
y Earl Williams 

William Tanner 

A Titl-e I Oral Language Specialist was employed in three of the 
Project schools. The OLS was prl-arlly responsible for providing an 
I expanded experient ial background to each of the^PCoject students. ^ 
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Specifically, the OLS developed and implemented the program's concept units 
in the ari^as of Food Marke t ing, J Communicat ion , T '"anspurtatiun , and Career 
/•warenesi. Ea-ch of the concept units was based upon either a classroom 

m 

experience or a field trip. Most of the work of the OLS consisted of small 

group or individual instructional activities with Project children. The 

following individuals served as 'Oral Language Specialists in the Program: 

Dorothy Poore 
^Pamela Grice 
Ruth Maroy 

One Title I liteading Specialist was employed at each of the four. 
Project schools. The primary function of the Reading Sp^cialisl: was to 
supplement the reading instruction provi'de'd by the classroom teachers.- 
The Reading Specialists generally warked with small groups or individual 
students on a regularly scheduled basis. In general, the Reading 
Specialist reported the use of DISTAR, Language Master, System 80, Audio 
Reading Progress Lab and SRA, Palo-Alto, and Merril linguistics materials.^ 

The following- itidividuals served as Reading Specialists in the Program 

Jean Decker 
Toni Christensen 
Pat Higgs 
Ethel Brown 

Kay Tenhoff ^ 
Diana Lowther 

University Year for Action Students ; Under the UYA program, students^ 
from Alaska Methodist University served as aides within the Program. 
Sp'ecif ically , UYA students assisted classroom teachers and the Title I 
specialists. The assistance was provided on a planned basis within each 
school. 

The students who participated in the p-rogram this year are: 

Margaret Davidson 
Robert Kerns 

Phillip Stevens . . j 



Home School Coordinator : Major duties of the Home School Coojrdinators 
centered aboup the establishment of communication between the school, the 
home and the community. In this* regard, Home School Coordinators not ^nly 
worked with individual children and ..parents , but also established local 
Parent Councils in each of the Title I schools. 

Home School Coordinators during the past year were: ' . 

Joel Davis 
Barbara Weil ^ 
Irmajean Barta 

Esther Grime^. ■ ■ 

Project Aides ; The work of ' the classroom aides entailed the following 
general • duties : " . 

s (a) assisting wit-h individual and small., group instruction 



(b) preparing teaching mat^erials under* ttie direction of the 
classroom instructor 

(c) maintaining records as directed 

The following individuals served as aides during the past year: 

T-1 Aides 



North Star 

Barbara Olson 
Joseph Ana^ews 
Betty Vogt / • 
Ethel ThonrS"s 
Terri Peterson 



Mt View 

Ernestine B^rney 
Hermina Boukamp 
Susan Caliban 

Jackie Magwood Doris Holden 
Linda Garrett » 

Sybil Bingham 
Vera Evans 



Denali 
Mary Scott 

Richard Jones - Susan Snyder 
Geneva Penatac 
Janice Kuykendall 
Bibiana Pezzinik 
Margaret Ollst^ad 



Fairview 



Phyllis Bowie - Wilraa Price 
Britt Ostby 
Mindy Fisher 
June Macon 

-Dorothy Smith - Sharon Walters 

Therese Sheehan 

Marion Hedburg - Willie Evans 
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Sauthvest Cooperativ e. ?:ducat ional Labocatbry (SWCEL) : 

: — ^ : : ~ • X . 

3Jtaff :-.er.bers fron the, SoutKwes c Cooperative Educational Laboratory 

i ' . " • 

provided several typ«s of technical assistance to the Preject. Initially, ^ 

the SW£EL staff proyided training in the area of test administration and^ v* 

the use. of the SWCEL Oral Language Program. In addition, 5WCEL provided 

training for the Qual ity^ Assurance Specialists (QAS) on the staff. 

Subsequently, SWCEL staff members ma'de three, onsite visits and provided 

test scoring (and interpretation) services to the program. • 

Northwest Regional Educational Laboratory (NWREL) : 

Staff members &Tom the Audit and Evaluation Section of the Northwest 
Regional Educational Laboratory provided technical assistance in the 
firm of third party evaluation services to the Project (see Section C) . 

E. Training of the Staff 

During the past year, a variety of specialized ' training opportunities 
was provided to the program staff members. The training program 
included both preservice and inservice offerings. 

The training was staged: (a) locally (i.e., within participating 
schools), (b) centrally (i.e., with staff members meetiug as a total 
group within the district) , and (c) out of state. A summary of Project 
training activities has been provided in Table II. 
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F. Chronology of Major Events 

In order to facilitate the reader's understanding of Project* 
" activities during the past year, a summary of major Project events 
has been prepared. The summary has been incorporated into Table III, 
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TABLE III ^ 

Chronological List of Majpr Events 

Anchorage Borough School District Title I Program 

1973-74 



SubmislHi of FY 74 Proposal 

Activation of ProBT^am 

Quality Assurance Specialist Workshop 
(SWCEL) 

Hiring of Teacher Aides 

OLP Preservice Workshop (SWCEL) 

Beginning of School 

Testing (SAT and Metropolitan Readiness) 

Student Identification/Selection 

Program Instructional Components 
All Activated 

'NWREL Brief Accomplishment Report //I 

Formation of Central Parent Council 

State Department of Education Onsite I 
Review 

Evaluation Workshop at NWREL in Portland 

NW!i^ Brief Accomplishment Report #2 

Federal Programs Workshop 

Second Quarterly Status Report Submitted 
to SDE 

Student Needs Survey Conducted (FY 75) 
Planning and Writing of Proposal (FY 75) 

20 



May 1, 1973 

July 1, 1973 
I 

August 6-10 
August 13 - 17 
August 20 - 24 
August 29 

September (f irstil^ek) 
September (first 3 weeks) 

September 30 
October 
December 6 

December 10 v 11 ^ 

December 13 - 14 

January 16 - 19 / 

February 19 - 22 
* 

February 1 
February 
February - June 



24 



TABLE III (Continued) 

Summer ^School Propo3al • Submitted April 1 

NWREL Brief Accomplishment Report #3 April 8-10 

FY 75 Proposal Submitted to SDE May 1 

Thi^d Quarterly Status Report Submitted 

to SDE , May 1 

USDE and SDE Onsite Revi^ew April 30 - May 1 

Final NWREL Evaluation Report June 12 - 14 

SWCEL Technical Assistance Visits November 6-8, 

February 20 - 22, and 
May 22 - 23 
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IV 

PROJECT OUTCOMES 

The results of the program's first year of operation fall into four 
categories. The first category is the impact that the program has had 
upon the c^ildreri who wefe involved. The second category consists of 
program' rnitpb»ts, that is, "self-standing," processes or documents which 
can be used by indiviq^uals in other settings. The third category focuses 
oti the insights which the staff .have developed as a result of operating 
the program for a year. The last category is represented by the attainment 
or nonattainment of Project objectives. In the sections which follow, each 
<jf these categories of results will be detailed. 

A. Effects of Program on Target Population 

Within this category of results, three varieties of data have been 

4 

collected from Project oLudents. These data are derived from student 
performance in three areas,, i.e., performance on the four concept area 
tests, perforinance on the SWCEL Oral Language Test and ^^^ormance on 
standardized reading test^. In, the paragraphs which follow, summaries 
of the findings in each of these performance areas will be presented. 
In order to simplify the presentation of data in each area, ,a topical 
approach to the display of data has been adopted. In particular, key 
issues within each perfofl?mance area will be identified and the data 
relevant to each issue will be presented." 



stude:;t performA::ce in the- four concept areas 

a. The Number of Students Tested : During the past year, major 

activities of the program ^centered about four concept 
areas; these areas were: Food Marketing, Communications, 
Transportation and Career • Awareness . Instructicma^ 
activities within each of these areas were based upon 
locally developed lessons. In general, the 41 lessons, 
which were produced by the project staff during the 
past year focused on field trip experiences. 

Assessment of student . performance in the four concept 
areas was accomplished by means of four oral tests. 
The tests were initially developed during the year and 
were tried out. in the winter term. Subsequently, the 
tests were revised and were administered on a sampling 
basis during the spring tenp» A sampling basis was 
adopted due to the amount of time required to administer 
the tests on an individual basis. Within the sampling 
plan, participating students in each Project classroom 
were randomly assigned to one of three groups. Subse- 
quently, two teBts (the career area test and one other) 
were individually administered to the students. A 
summary of the number "of Project children to whom 
tests were administered is provided in -TablailV. 



TABLE .IV ^ ; 

V 

NUMBER OF PROJECT CHILDREN 
WHO WERE TESTED IN EACH CONCEPT AR^ 

CO NCEPT AREAS / ■ , 

7 tTareeT 

■ Food Communication Tr ansportation Awareness 



17. 15 ^ 14 46 

14 ' • 16 13 . 43 

14 17 . ^ 22 53 

15 14 ' ^ . 14 43 



b. 




The Number of Students Who Achieved the Q^J^^^^P I^^ Specific 

performance objectives in the concept alHH|s called for 
75 percent of the students to respond cc|iBectly to 
90 percent or more of the items in the IwR'^s of food, 
transportation and communidation. The o^ective further 
called, for 80 percent of the students to respond correctly 
to career awareness items. In Figiires 2-17, the percentag^e 
of spudents in grade levels K-3, who responded at various 
levels of performance on the concept tests is portrayed. 
In/addition, the percentage of students who met or exceeded 
tiiie'' criterion level is also presented. 
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A review of Figures 2-5, reveals that 19-,* 21-, 64- 
and 87 percent of the children in grades K~3, respectively, 
achieved the criterion level on the test dealing with 
food marketing. With^ reject to the comnianications test, 
zero percent, 38-, 47- and 50 percent of the respondents 
in grades K-3, respectively, performed successfully. 
Relative, to the test dealing with transportation, the 
attainment figures for children in grades, K-3, were: 
seven per'cent, 38-, 64- and 71 percent, respectively. At 
this point, two^trends seem evident: first, the percentage 
of children who attain the criterion on these three Cests 
generally increases at each successive grade level. 'This, 
then, implies that a maturational variable is operating. 
Stated differently, test items which represent a 
challenge to the Kindergarten child may not be a challenge 
to the third grade child\ This suggests that a hierarchy 
of concepts in each of the areas might prove useful in 
organizing the lesson plan material. 

A review of Figures lA-17, indicates that 27 percent of 
Kindergarten children, 12 percent of the first grade 
' children, four percent of the second grade children arid 

37 percent of the third grade children achieved mastery 
in the area of career information. It should be noted 
that "soft" and "semi-soft" rules were applied in ^coring 
the responses of the children in grades K and 1. Thus, 
comparisons of the relative performance of children in 
various grade levels is not entirely appropriate for these 
data. The soft ^nd semi-soft- rules for scoring, 
incidentally, were brought about because of the legitimate 
concern of teachers that the youager children were not at 
th^ level which would permit them to verbalize as well as 
the older children. Hence, the soft scoring rules called 
for the awarding of credit for a response which was 
descriptive of a role task rather than a role label.* 

/ The percentage of children who achieved the objective 

at each grade level, reg^ardless of the scoring procedures 
is not high. Thus, a thorough/^reconsi deration of the 
concept areas, as well as the supporting instructional 
procedures is recommertded. 



* For example, in response to the question, "What is a job you could have 
if vou worked at an airport?", a Kindergartea child would be accorded 
full credit if he responded, "take tickets.'; A thirc? grade child, however, 
would be expected to respond with a job name or label (Ticket Agent). 
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ANCHORAGE FIGURES 2-3 

.Distribution of Student Attainment 
Total Scores on. Food Marketing Items 
1973-74 
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ANCHORAGE FIGURES 4-5 

Dist r ibut i'on of Student Attainment 
Total Scores on Food Marketing Items 
1973-74 

90% 
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' Di;i t r ibution o: Student Attainment 
Total "Scpres on Communication Items 
1973-74 
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ANCHORAGE FIGURES 8-9 
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ANCHORAGE FIGURES 



Distribution of Student 



Total Scores on 

1973-74 
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ANCHORAGE FIGURES 12-13 

Distribution of Student Attainment 
Total Scores on Transportation Items 

1973-74 \ 




AilCHORAGE FIGURES 16-17 

Distribut^ion of STudfent Attainment 
Total Score on Career ihfonaation 




4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 



Grade 3 

(k FIGURE 17 

ERiC . . 



1: 



r 



c. Ttie Average Attainments of Project Students on the Four 

Concept Testsf In addition to tabulating the number of child- 
ren wiiO att^iiiea tne mastery level of each concept ' 
tegt, another analysis is afforded by examination of 
the average attainment of children on the test. In 
Table V the number of items on each test and the 
average percent correct (per grade level) is presented. 
The average grade three performance on the food 
marketing test, for example, "was 95 percent torrect ' 
responses. On the othrt^iand, the average percent 
of correct responses by»the Kindergarten children to 
the 17 co^unication items was 55 percent. Again, the 
figures in Table V indicate successively higher per- 
formance at& each grade level. Thiis, the question is 
raised: AreVthe test items sufficiently challenging 
\to the oldeif children? Staff consideration of this 
issue is recommended. ' ^ 



TABLE V 

AVERAGE PERFORMAi^CE OF PROJECT CHILDREN ON CONCEPT TESTS 
. ' * \ 

CAREER 

FOOD COMMUNICATION TRANSPORTATION AWARENJISS 

n- = 23 n = 17 n = 19 n = 19 



//of • % Mean //of % M^n //of % Mean 7/ of % Mean 
GRADE Children/Attained Children/Attained Children/ Attained Children/Attained 



K 


17 


73% 


15 


55% 


14 


51% 


46 


55% 


1 


14 


73 


16 


65 


13 


78 


43 


57 


2 


14 


92 


17 


82 


22 


84 


53 


87 


3 


15 


95 


14 


87 ^ 


14 


92 


43 


72 



d. The Difficulty L^vel of Specific Test Items ; Specific areas 
in whiQh. children encountered difficulty (as well as 
ease) in responding to the tests is provided in Tables Vla-d. 
Specifically, the percentage of children who responded 
incorrectly to each test item is provided in Tables Vla-d. 

As evidenced in Tables Vla-d, most items follow the 
trend of being more difficult for the younger children 
than for older children. For example, Item number 17 
on the food marketing test was missed by 31 percent 
of the Kindergarten students, 12 percent of the first 
grade students, six percent of the second grade 
students and by none of the third grade students. 
These findings again suggest organization of the 
curriculum along hierarchical lines may be fruitful. 
The data presented in Tables Via-d also suggest 
^ particular areas where additio^l curriculum instructional 
support is needed. 
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ANCHORAGE 

TABLE Via 
Item Analysis: Concept Test 
Percentage of Children U^o Responded Incorrectly 

Oa Food Marketing Items / 
1973-74 



Item ' [ Percentage Wrong by Grade Level 

K I II *III 





* 


rt = 16 


n = 14 


n = 14 


n = 15 


1 












2 




25% 


6% 


— 


— 


3 


% 


38% 


12% 


— 


' 13% 


4 


25% 


43% 


6% 


13% 


c 




6% 











6 




38% 


12% 


12% 




7 




6% 


— 


, — 




8 




, 6% 








9 




25% 




12% 




10 




25% 


36% 


12% 


13% 


11 




13% 


6% • 




7% 


12 




31% 


18% 


6% 


7% 


13 




38% 


/ 50% 


12% 


7% 


14 




19% 


24% 






15 




6% 








1'6 . 




13% 


6% 






17 




31% 


12% 


6% 




18 




19% 


24% 






19 




38% 


43% 




■ 7% 


20 




50%- 


71% 


6% ' 


i3% 


21 




• 31% 


64% 


18% 




22 


* 


56% 


64% 


24% 


7% 


23 




100% 


93% 


50% 


27% 



( 
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TABLE VIb ^ 
' Item Analysis: Concept Test 
Percentage of rhildren Vlho Responded Incorrectly 
on Connnunication Items 
1973-74 



Item 


Percentage Wrong by Grade 


Level i. 


• 


K *• 


I 


II 


Ill 




n =• 15 


n = 16 


n = 17 


n = 14 


1 


207. 


19% 


6% 


— — ' 


z 


47% 


31% 


• 6% 


6% 


3 


80% 


50% 


24% 


18% 


4 


47% 


38% 


29% 


6% 


5 


87% 


69% 


59% 


24% 




47% 


25% 


6% 


12% 


7 


53% 


19% 


6% 


6% . 


8 


40% 


38% 


1'2% 


6% 


•«. 

9 




13% 






10 


27% ' 


13% 


18% 


6% 


11 


• 60% 


31% 


24% . 


12% 


12 


20% 


19% 


18% 






67% 


38% 


29% 


24% 


14 


33% 


38% 


12% 




15 


20% 


13% 


6% 


12% 


16 


47% 


'19% 


24%' 


AL8% 


17 


73% 


44% 


41% 


26% 



ANCHORAGE 
TABLE Vic 

Item Analysis: Concept Test, ^ ^ 

Percentage of Children VTho Responded Incorrectly 
on Transportation Items 
1973-74 



Item 



Percentage Wrong by Grade Level 







r 

K 


I 


II 


III 






n = 14 


n = 13 


n = 22 


- 

n = 14 


1 




29% 


— — 


5% 


— 


2 




50% 




9%" 


-v^^ 5% 


3 . 




86% 


15% 


18% 


\ll% 


k 




36% 


15% 


5% 


X- 


5 




36% 


31% 


9% 


5% 


6 




71% 


54% 


23% 


11% 


7 




71% 


23% 


9% 


5% 


8 






46% 


32% 


16% 


9 




36% 


8% 


5% ■ 




10 




64% 


23% 


14% 


11% 


11 




7% 








12 




7% 






5% 


13 




50% 


15% 


14% 


11% 


14 I 




43% 


15% 
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64% 


23% 


23% 


11% 
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21% * 






» 
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43% 
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18% 


5% 
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64% 


31% 


55% 


16% (! 
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.50% 
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^ 5% 


5% 
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ANCHORAGE 
TABLE Md 
Iten Analysis: Concept €est 
^Percentage of Children Who Responded Incorrectly 
on Career Information 
1973-74 • 



Item \ Percentage Wrong by Grade Level 

















1 ^ 
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III 






r 




- 

n = 53 


n = 43 




n = 46 


n. 


= 43 


1 


49% 




28% 


34% 


9% 


2 


40%, 




117. 


74a 


60% 


3 


22% 




19% 


4% 


.2% 




60% 




42% 


13% 


2/4 


5 


Jb/i 




7 77 




J O/m 


6 


64% 




98% 


81% 


72% 


7 


22% 




63% 


* 81% 


53% 


8 


' 51% 




86% 


74% 


44% 


9 


80% 




84% 


85% 


56% 


10 


16% 




2% - 


4% 


2% 


11 


22% 




14%. 


4% 


5% 


12 






40% 


19% 


■■ 9% 


13 


78% 




lot 


43% 


30% 


14 


38% 




117. 


72% 


42% 


15 


71% 




93% 


70% 


58% . 


16 


1 ^ 4% 






4% 




17 


53% 




44% 


13% 


7% 


18 


29% 




12% 


2% 




19 


69% 




60% 


28% 


12% 
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2. STUDENT ATTAINMERT ON THE SWCEL ORAL LANGUAGE TEST 



a. The Number of Children Who Gained At Least 30. Points on the 

SWCEL Oral Language Test : The'SWCEL test was administered on a 
^ pre basis to all Title I children (grades K-l)during 
the fall term. In addition, Project teachers and 
specialists made judgments regarding the language 
capability of each\ child who was tested. Due to the 
delayed return of the pretest results, Project children 
were selected for participation in the SWCEL Oral • 
Language Program on the basis of staff judgments. 

^ 

The Project objective relating to performance on the 
SWCEL test call^ for a 30-point gain between pre and 
posttests for 80 percent of the participants. Ir 
order to permit comparisons, the pre and post perfor- 
mances of 84 control group students were also obtained. 
The control group was comprised of nonTitle I Children 
in grades K-1 in the North Star Elementary School. A 
summary of student gains on the SWCEL test is 
presented in Table VII. 

As indicated. in Table VII, 53 percent of the Kindergarten 
children and 64 percent of the first grade children 
achieved gains of 30 or more points. In contrast, the 
control group children achieved gains of only eight 
percent and 11 percent (grades K and 1, respectively). 
The proportion of Project children who gained at least 
30 points was significantly greater than that of the • 
control children (Oppenheim test, p < -05), at both the 
Kindergarten and first grade level. 



TABLE VII 



PERCENTAGE 


OF CHILDREN 


ACHIEVING 


30-POINT SWCEL GAIN 




GRADE 


Total 
Number 
Childrerf 




Number 
Children With 
30-Point Gain 


Percent 
Children With 
30-Pciint Gain 


Kindergarten : 










Target Group 


49 




26 


53% ^ 


Control Group 
* 


38 




3. 


8% 


First Grade: 










Target Group 


74 




47 


64% * 


# 

Control Group 


46 




5 


11% 


* Difference in percent is significantly 
(Oppenheim Test, 1966) 

4/ 


different at the ,05 


level. 
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The Average Performance of Project and Controly Students on* the 
SWCEL Test ; While the proportion of children who gaine'^^ the 

minimal number of points on the SWCEL test is pe\tinent, . 
it does not by itself provide a complete overview of 
student attainments on the SWCEL tasks. Another way 
' of reviewing group perf ormanc^es is to examine the means 
of the various groups on pre^^^nd posttests and to 
determine if significant differences exist between 
various group means. Tables VIII^ and Vlllb provide just 
such a summary. 

In Tables'VIIIa and b, the mean performance of each classroom 
group of Project children, i.e., those participating in 
SWCEL for th| entire year, and each classroom group of 
control children is presented. In addition, near the 
bottom of each table the total group means are presented 
agd statistical tests have been summarized. t 

The major findings presented in Tables Vllla-b are these: 

• both Project and control groups (K and 1) demon- 
strated a statistically significant improvement 
in their performances between pre and posttests. 

• based upon comparisons of pretest perfoirmance, the 
control students represented robust comparison 
groups. Specifically, the Kindergarten control 
students are not found to differ statistically on 
the basis of the pretest from their Project counter- 
parts. However, the'^first grade control students 
performed at a higher level on the pretest; than did 
the first grade Project students. This means (at 
least for the first grade Project students) that in 
order to demonstrate an absolute superiority in 
performance on the posttest, a substantial deficit 
would have to be made up and then additional gains 
would have to be demonstrated. 

> 

• comparison of the posttest performance of the 
Prpject and control students reveals that the Project 
students (Kindergarten level) performed at'a level 
which was significantly higher than that of the 
controls. That is, the two groups' began the year 

at essentially the same « level, but the P.roject 
^ students performed in a manner on the posttest which 
was significantly better than that of the controls. 

• comparison of the posttest performance of the Project 
and control students (first grade level) reveals that 
the Project students performed at a level which was 
significantly higher than that of the controls. Thus, 
the Project children made gains which: (1) overcame 

an earlier deficit and v(2) which resulted in a superior 
level of performance* s; 
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ANCHORAGE 

" TABLE Villa 

Pre and Posttest Means and .Standard Deviations 
on the SWCEL Test of Oral English Production (Kindergarten) 

1973-74- 



Group 



Pretest 



Posttest 



Control 1 



Control 2 



n = 22 



n = 16 



Mn 
S.D. 

Mn 
S.D. 



118.32 
25.33 

108.25 
27.48 



Mn 
i.D. 




S.D. = 



125.82 
21.18 

119.13 
30.66 



1 


n = 


9 


Mn 




99. 


Mn 




106.78 








S.D. 


m 


22.15 


S.D. 


a 


14.73 


2 


n = 


16 


Mn 




101.63 


Mn 




137.63 








S.D. 


r 


^22.43 


S.D. 


=3 


23.42 


3 


<n = 


10 


mJ 




99.6 






149.6 








s.dJ 




32.24 


S.D. 




33.01 


4 


n = 


14 


Ma 




119.43 


Mn 




145.93 








S.D. 




25.08 


S.D. 




25.43 



To-tal Control 



n = 38 = llA.t)8 

S.D. = 26.38 

Pre vs. Post: 



t 

df 



3.2446^ 
37 



Ml 
S.D. 



123.0 
25.43 



Total Project 



n = 49 Mn = 105.81 

S.D. =» 26.09 

Pre vs. Post: 



Mn 
S.D. 



136.78 
28.54 



t 

df 



10.4217* 
48 



Project vs. Control: Pre . ?ost 



t = 1.46 t = 2.34* 

df = 85 df = 85 



* p .05 



ANQHORAGE 
• TABLE VI,IIb 
Pre and Poattest Means and Standard Deviations 
on the SWCEL Test of Oral English Production (Grade 1) 

1973' 



/ 



orouo 



Pretest 



Posttest 



Control 


1 


n 




12 




Mn 




130.08 


- Mn 




146.75 














S.D. 


= 


19.92 ^ 


, S.D. 




13.87 . 


Control 


2 


n 


= 


9 




Mn 




129.33 


Mn 




137 














S.D. 




25.12 


S.D. 




17.51 ^ 


Control 


3 


n 




12 




Mn 




137.5 


Mn. 




148.67 ^ 














S.D. 




24.21. 


S.D. 




24,37 


Control 


4 


n 




13 




Mn 




138o31 


Mn 




149.38 














S.D. 




17.59 


S.D. 




22.45 


Project 


1 


n 




12 




Mn 


: 


116.75 


Mn 




149.83 














S.D. 




20.22 


S.D^ 




23.46 


Project 


2 


n 




8 




Mn 




114.13 


Mn 




138.75 














S.D. 




27. 8 


S.D. 




24. 77 


Project 


3 


n 




16 








130.13 


Mn 




161.31 














S.D. 




24.94 


S.D. 




26.03 


Project 


4 


n 




17 




Mn 




135.53 


Mn 




174.59 














S.D. 




19. 83 


S.D. 




14.76 


Project 


5 


n 


= 


3 




Mn 




131.33 


Mrl 


= 


154.67 














S.D. 




19.74 


S.D. 




17.95 


Project 


6 


n 




8 




t Mn 




126.88 


Mn 




172.25 












S.D. 




16.58 


S.D. 




11.87 


Project 


7 


n 




4 




Mn 




116. 


Mn 




147.25 












S.D. 




25.15 ; 


S.D. 




12.61 


Project 


8 


n 




6 




Mn 




114. 67| 


Mn 




142.83 












S.D. 




33.50' 


S.D. 




16.83 



Total Control 



n = 46 



Mn = 134.13 Mn = 146.09 

S.D. = 21.57 S.D. = 20.07 

t = 5. 37* 
df = 45 ♦ 



To&al Projact 



b!n = 125.15 t-i - 15r.. 72 

S.L. = 23.66 ' S.D. = 23.29 

t = 16. 18 * 

df = 73 



Project vs. Control: 
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p < .05 



Pre 

50 ^ ^ -2.09* 

df - 113 



Post 
- t = 3.04* 
,df = 118 
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Percentage of Students who Scored 130 Points or More on the 
SWCEL Tests; Within the SWCEL Oral Language Program, children 

are classified according to their ability to use language 
appropriately. The SWCEL, test, in fact, provides the 
basis for such classifications. A key area within the 
range of possible scores on the SWCEL test is the 
130 point level. According to the current operating 
procedures, children who score below this point are 
recommended for inclusion into the SWCEL ins tructionalJl^ 
program. 

Critical questions for the Project then become: How 
many of the Project children are above this level when 
they begin and how many are above this level when they 
have finished a year's instruction? Another question is: 
what happens to the language performance of children 
who are not provided with the SWCEL program? 

Answers to these questions are provided in Table IX. 
In Table IX the percentage of students in both Project 
and control groups who exceeded the critical level on 
both pre and posttests are presented. Examination of 
the two s6ts of control groups (K and 1) indicates 
that without SWCEL instruction a gain of nine percent 
and 28 percent in students scoring aboVe the critical 
level occurred. On the other hand, the two Project 
groups (K and 1) contained 45 percent and 56 percent more 
children above the critical level at the time of the 
posttest. These findings would seem to indicate that 
the language performance of the nonTitle I children 
improved somewhat without the SWCEL program, but that 
use of the SWCEL program accelerated the growth 
of language performance to a considerable extent. 



V 
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\ 
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TABLE IX 1 

' ' \ \ 

.Percentage of Students V/ho Scored 130 o^ More 

on Pre and Post SWCEL Test \ 

1973-74 \ 



Total N 



Percentage 130 + 
Pretest 



Percentage 130 + 
Posttest 





22 




(5) 


23% 




I 


(11) 


50% 




Control 2 


16 




(3) 


19% 


Total = 


38% 


(6) 


38% 


Total 


Project 1 


9 




(1) 


11% 






(0) 


Q 




Project 2 


16 




(0) 


0 






(12) 


^ 1 
7^% 




Project 3 


10 




(2) 


20% 






(8) 


Boi^ 




Project 4 


14 




(5) 


38% 


Total^= 


16% 


■ (12) 86^ 


Total 


CrAde I \ 


Control 1 


. 12 




(4) 


33% 






(9) 


75% 


\ 


Control 2 


9 




(4) 


44% 






(6) 


67% 


\ 

\ 

\ 


Control 3 
Control 4 

> 


12 




(8) 


67% 






(9) 


75% 


\ 


13 

a 




(7) 


54% 


Total = 


50% 


(12) 


92% 


To^^al 



= 45% 



6l/^ 



78% 



Project 


1 


12 




(2) 


17% 


(10) 83%s 


Project 


2 


8, 




(2) 


25% 


(6) 75% 


Project 


3 


16 




(6) 


38% \ 


(15) 94% 


Project 


4- 


17 . 




(10) 


59% . 


^ (17)100% 


Project 


5 


3 - 




(2) 


67% 


(3)100% 


Project 


6 


8 




"^^^ 




(8)100% 


Project 


7 


4 




(1) 


25% 


(4)100% 


Project 


8 


6 




(2) 


33% 


\ (5) 83% 



Total = 36% 



To\al = 92% 
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d. Gains -and Losses on the SWCEL Oral Language Test: As indicated 
in earlier paragraphs, both Project and control children 
made notable gains in their perfomance on the SWCEL 
Oral .Language Test. l/hile this statement is generally 
true, there were instances in which the scores of child- 
ren declined from the pretest to the posttest. A 
summary of both the gainsr an'd losses evidenced by Project 
and control children is presented in Figure 18. 

As noted in Figure 18, the gainb made By both the Project 
and control students easily outweigheti the losses. . In 
addition, the gains' made by the Project children (both 
K and Grade 1) were greater than the gains made by the 
control chilflren. ^ On the other hand, approximately fiye 
Project children in- the Kindergarten and four Project, 
children in the First Grade evidenced a lower level 
of performance on the posttest than on the pretegt. 
Such losses are diffl^lt to e^cplain in light of the 
concentrated instruction which was provided to these 
children over the course of the year. However, at 
least three possibilities exist: 



ERIC 



(1) the pretests overestimated the initial performance 
of the -children (or the^osttest underestimated 
the final performance) ^ ^ 



(2) 



the instruction was not provided tA the extent 
(or in. the manner) £hit tfye monitoring records 



indicate 



(3) the instruction is not 'uniformly beneficiaL to 
all Projept students. . 

In' terms of explanatory facts, a review of those in- 
stances in w^ich losses were observed indicates that: 
two of five losses at the Kindergarten level and three 
of fbur*losses at the First Grade level (or a total of 
56% across two grade levels) were associated with 
children who scored at or aboye the.'^L^G point level 
of the pretest, "fhis then suggests^ that a ^egression 
effect may be in operation. That .is, in over half the 
cases where Project children recorded lower scores ^ 
on the posttest then on the pretest, the initial test 
scores were ^elativeljj high. 




00 



a 



o 

C 
O 



CO 

a 

H 

<U C 
S '"^ 



u o 

3 a 

W r-l 

;^ cj o 

o c , 

u , 

jiM O CO 



•H 


CO 


4-1 


a 


c 


a 




o 






-H 




o 


4-1 


o 




CTJ 




t: 


4J 


CU 




CO 


u 


ni 


fo 




•H 




CO 


Adra 







o 



□ 



a 

< 
a; 

H 

cn' 
a: 



1 1 1 1 ir. 



rrm 




o 
I 



o 
I 



o 



<u 
c 

a 



o 



CO 

o 



5> 

o 



\ \ \ > * I L_ 



c^j o 00 o <r 



O'OO vO -^" OOO O ^ CNI 



U9apiTq"^'40 joqmnu 



ERIC 



54 



1 



THE PERFORMANCE OF PROJECT CHILDREN ON STANDARDIZED READING 
TESTS 



Improved reading performance is a major thrust of the 
present project'. In fact, much of the instructional activity 
devoted to the acquisition of basic concepts and language fac- 
ility was undertaken because of the presumed linkage between 
language and reading. Thus, student performam:e on reading 
tests is a„matter of importance to the project staff. 

The reading performance of children participating in the 
District's Title I program for the last two years was used to 
provide a comparative base for the present study. Use of these 
data was justified on the grounds that: j 

(a) the control children were largely enrolled in the same 
schools as the current project children; 

(b) no othfer known group was so similar to the current 
prS^ect children; 

(c) the present total Title I program represents a substantial 
change from that which was previously offered; 

(d) the data were available. 

In analysing the reading data from the current year, only 
complete or "paired" data were used. That is, only data from 
project children who had been in the -Title I program for the 
entire year were used. A similar rule was also adopted in . 
compiling data from the students in the baseline years (i.e,, 
1971-72 and 1972-73). 



a. ' Performance of the Project and Baseline Children Throughout 
the Year: Because difff^rent fonss cf the District's selected 
tests ^were used during the pre-interim-post testing 
phases, no direct, statistical comparison of student 
'■growth" is possible. However, a fair approximation 
of average student performance across the yeai^ is 
provided in Table X. 




TABLE X 

Average Student Performance of Stanford Readiness/ 
Achievement Tests: Baseline and Project Children 

i PRETEST INTERIM TEST @ POSTTEST 

MN N SD MN N SD MN N SD 

GRADE LEVEL . 

First Grade * Metropolitan' , SAT Primary I (X) 

Readiness 

Project 1 58.20 55 16.33 — " — ' 19.24, 55 6.72 

(1.7)* 

Control 1 . 51.42 117 13.04 -- — — 15.95 117 6.40 

(1.6)* 

Second Grade SAT Primary I SAT Primary II SAT Primary III 

(W). (X) (Y) 

Project 2 14.53 59 4.36 12.32 57 6.17 14.19 59 7.30 

(1.5)* (2.1)* (2.5)* 

Control 2 15.21 116 4.48 — -- — 13.24 116. 5.30 

(1.5)* (2.-3)* 

Third Grade SAT Primary II SAT Primary II SAT Primary II 

- (W) (Y) (X) 

Project 3 11.53 66 5.05 15.21 61 5.29 18.80 66 5.25 

(2.1)* (2.6)* (2.9)* 

Control 3 11.71 106 5.74 __ __ __ 18.60 106 5.97 

(2.1)* , (2.9)* 

.*Grade Level Equivalent 
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As noted in Table X, children in the Project group appear 
to progress moderately in their reading attainments through- 
out the year. Note for example, that on the average, 
Project children in grade two began the year at 1.5 
(grade level equivalent) in reading, they progressed to 
the 2.1 level by the time of the interim testing and 
completed the year at the 2.5 level. A similar pattern 
is evident for the third grade children- .In both 
instances, the children began the year somewhat behind 
the average or expected level, gained between eight and 
ten months, but were still behind when they completed 
the year. In fact, it appears that the end of the year 
deficit becomes progressively worse as the grade level 
increases. 

In considering the gains made by^^wie Project children 
and the control children, no substantial improvement in 
performance is evidenced. For example, the apparent 
gain made by third grade Project children (see grade 
level* equivalents) was virtually identical to the gains 
made by the children in the control group. These findings 
suggest that concerted review of Project instructional 
efforts in the area of reading would be appropriate. 

The Performance of Subgroups of Project Children : In furthering 
the present analysis, data from Title I children (1971-72 
and 1972-73) were categorized along two dimensions. In 
particular, the control group data were divided into 
three groups according to pretest performance. The 
groupings were made roughly by thirds- Subsequently,' 
posttest data from the same students were categorized 
again by thirds- The resultant 3X3 tables provide the 
basis for establishing expected levels of performance 
for children ^in the present Project* The 3X3 tables 
are similar to actuarial or "experience" tables used 
by life insurance companies- In effect, the tables show 
that if an individual performs initially at a-certain 
level, his chances of performing subsequently at another ^ 
level are known- In Tables Xla to Xlc, the 3X3 actuarial 
tables which have been calculated for the baseline groups 
are exhibited^ In Table, ^la (first grade), for example, 
^ .13 percent of the children in the baseline or control 
group who scored in the upper third on the pretest, 
scored In the lower , third of the distribution on the posttest 
However, 68 percent of thoae baseline children who scored 
in the lower third on the pretest also scored in the lower 
third on the posttest- 

On the basis of the "experience" gained with the control 
students, it is of interest to note how the cHildren in 
the Project group performed- 
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First Grade Children-Upper Category 

The Project children who performed in the upper range on 
the pretest (see top line, Table Xla) were found to be 
superior in their performance on both the pre and posttests. 
This implies that the reading program facilitated their 
learning. 



First Grade Children-Middle Category 

The Project children who performed at the mid level on the 
pretest were not essentially different from their control 
counterparts. While there was some apparent "shifting" 
in performance on the posttest. (i.e.*, fewer of these 
children performed at either the upper or the lower level 
of the posttest), there were n£ overall changes of 
statistical significance on the posttest. 



First Grade Children-Lower Category 



There was no statistical difference between the baseline 
children and the Project children on the pretest. 
However, the data indicate chat a number of shifts occurred 
^on the posttests. That is, fewer of the children in this 
category remained in the lowest category on the posttest. 
Additionally, the number of baseline childreJf who scored 
in the middle category on the posttest (21 percent) was 
less than that expected (i.e., 26 percent for baseline 
children). Finally, 36 percent of the children in this 
category performed in the' upper category on the posttest. 
This figure is in sharp contrast to the performance of 
their baseline counterparts, only five percent of whom 
fell into the upper range on the posttest. 

Overall;' the first grade Project children performed better on the 
pretest, but this must be largely attributed to the children in 
th^ upper group. In examining the overall post performance of 
the Project and baseline groups, the superiority of the Project 
children is again evidenced (t - 3.10, df = 170, p<i05). 
However, as noted above, this superior performance must be largely 
attributed to: 

••(l) the initial superiority of the Project children in the upper 
group which was maintained by the program. 

(2) the children in the lower group who were aot significantly 
di^fferent from their baseline counterparts on the pretest, 
but who were significantly better on the posttest. These 
findings inply that the District's reading instruction was 
inc;^t ^^f fective for the children from thex low (pretest) group. 
The findings also imply that the- middle group could profit 
from additional instruction. 
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Secend Grade Children-Upper Category 



The second grade children who performed in the upper 
category on the pretest were not significantly different 
from their baseline counterparts. (A» t-test of the 
difference between the group means on the pretest yielded 
a nonsignificant result.) 

Ori the other hand, the performance of the Project children 
tended to shift to the upper level on the posttest. While 
this apparent shift was encouraging, it was not statistically 
significant. ^ 



Second Grade Children-Middle Category 

Differences between the Project and baseline childiren in 
this category initially favored the baseline group (t = 
2.51, df^= 7F, p^.05). "However, Tia differences between 
the two groups were noted on the posttest. In a sense, 
then, the Project children 'In this group overcame an 
initial deficit. This, then, represents a "credit** for 
the Project. 



J. 



Second Grade Children-Lower Category 

•} — ' ' 

Again, there were no statistical differences between 
Project children and baseline children on either the 
^ pretest or the posttest. However, it was in this 

category that the more dramatic shifts^ within the second 
grade occurred. As noted in Table Xlb, only 27 percent 
of the Project children, in this category (as opposed to 
59 percent of the baseline children) performed in the 
lowest category on the posttest. Additionally, there 
were upward shifts into the other two categories on the 
posttest. Thus, it appears that the lowest group as a 
whole exhibited the most uniform pattern of gain of all 
the second grade Project children* 

Ovefall, the performance of the second grade Project children 
was not 'significantly different from that of the baseline children 
on either the pretest or the posttest. On the other hand, the 
middle' range Project rhildren overcame an initial lead held by 
their baseline counterparts. 

These findings imply that the reading instruction provided to the 
Project children during the past year did not represent a marked 
improvement over the program which was provided during the baseline 
years. 
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^ . Third Grade Children-Upper Category 

The pre and post performances of Project children and 
baseline children in this category were not found to 
differ significantly. Examination of Table Xlc indicates 
that a smaller percentage of Project children than * 
baseline children performed in the upper level of the 
posttest (i.e., 41 percent vs. 70 percent). In addition, ; 
a lightly greater percent of the Project children performed ^ 
at a lower level on the posttest (18 percent vs. 13 percent 
for the baseline group). Thus, the general trend for this 
group of Project children was toward a lower level of 
performance. 



Third Grade Children-Middle Category 

The overall pre and posttest difference for children in 
this category did not distinguish between Project children 
• and baseline children* The only apparent **shif t" in 
performance in this category was from the lower to the 
middle category of performance on the posttest. 



Thircj Grade Children-tower Category 

Differences between the performance of baseline and Project 
children on the pretest initially favored the Project 
children (t = 2.03, df = 64, p<C-05)* Further analysis 
of the posttest data for these groups indicated that the 
Project students generally "shifted" to the middle and 
upper categories of performance on the posttest. However, 
the shifts were not of sufficient magnitude to yield 

statistically significant dif*erfinces for the entire 
group on the posttest. 

•• I- 
Overall,^ no statistical differences between pre and posttests of 
the Project and baseline third grade students emerged. This 
finding suggests that the Title I reading program for the third 
grade students was no more effective than it had been during 
the baseline years. 
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Project Outputs 

According to staff members, the Anchorage OLP has 'yielded a number 
of products and processes which have potential for application in other 

P 

settings. Specifically, a number of documents have been produced and 
a number Of processes have been sufficiently delineated so that their 
examination by other parties interested in duplicating the present 
effort is possible. 

These processes and documents are enumerated in brief: 

1. Forty-one lesson plans have been developed in the areas of Food 
Marketing, Transportation, Communications and Career Awareness. 

f Jhe lesson plans contain both cla^^room and field trip activities 
and are suitable for use with children in grades K-3. 

2. The staff has developed a monitoring and reporting system which 
is useful for managing a multi-grade, multi-site program. 

3. The staff has developed role models for Ijoth Oral Language i 
Specialists and Home-School Coordinators. The job descriptions 

of these and other Project personnel have been incorporated into 
the Project proposal. 

4. The* staff has developed descriptions of the instructional aspects of 
the program. Collectively, these descriptions constitute the 
Project's Instructional Model . 

5. A Projiect Specification Sheet has been prepared by the Project 
Evaluator and staff which describes the major facets of the 
program. 
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Informal Results 



Because of the continuing and cumulative nature of the Title I 
Program, it becomes important to document the experience of the program 
staff on a periodic basis. That is, the insights which are developed 
by the staff during the operational ye^r represent an important source 
of data for program modification; accordingly, such insights should 
be preserved. Within the present^ section, an opportunity has been 
provided for the staff to summarize their own observations from the 
past year. 



Staff Insights : 



The staff generally felt that the management reporting 
system was generally s^Mtisf actory . However, there is 
a need to alter some of the Project's internal reporting 
forms. For example, indiv:idual student records should 
be consolidated on a single form .* 

There is a need for regularly scheduled meetings with 
principals, teachers and specialists. ^ 

Program training for next year should include: 

(a) A more thorough orientatibn to the entire program for 
the entire staff. I 

(b) Inservice training for program specialists prior to 
the opening of school. 

The Oral Language Specialist^ could profit from a scheduled 
workshop prior to the start of school. The workshop should 
focus on the devefopment (and redevelopment) of the concept 
lesson plans and concept area tests. " ^ 

Staff members feel that a more consistent approach to 
planning local parent council meetings would be appropriate. 

Intra-district ^dissemination efforts should probably be 
strengthened. This is particularly important in schools 
which represent potential expansion sites for Title I. 
Additionally, such an intra-district dissemination effort 
would forestall the duplication of consultant services. 



Each category of staff specialists should be "cross-trained" 
through participation in the training activities of the 
other specialists. This would result in cooperative efforts 
and understanding within the program, 

\ 
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D. Status jf Project Obiectives 



/ 



The .present s4ction of the report deals with- the sta^ui of all 

I 

Project objectives at the end of the second operational year^ In 
order 'to facilitate the display of information, a tabular arrangomen^ 



, has been devised. The resultant table contains the following informa- 
tioa: ' * • 

1. Brief identification of each objective ( 

2. 'Statu* of each objective (i.e., whether or not 

the abjective was attained, attained .in part, 
not activated, etc.) ^ 

f ~^ . 

3. A summary of recent activity performed in behalf 
of th^ objective- 



4. The location of the documentation which subs tan ti'&tes 
the claims made ift behalf of the bb'jective 



'bince similar tables have been used in the preparation of the 
ProJ^t'i Brief Accomplishment Reports, the table also serves to document 
-the history of^ach objective over the four reporting periods of the 



r 



pper&tional year. 
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Management Component Overview 

There are sixteen objectives in the Project's Management Component, 
One of these, objective p. delating to the Home-School Coordinator, 
was established -in January 1974. 

Of the sixteen objectives, ten were attained by the time of the 
first Brief Accomplishment Report in Ocfober. These ten objectives 
called for: i 

• The establishment of an Advisory Council. 

• "Establishment .by the Advisory Council of student priorties. 

• The identification of target population and school sites. 

• The" establishment or program performance objectives. 

• The development of,, the Evaluation Plan. 

• Th^ development of the program proposal. 

• The -'submission of the proposal. *• 

- _ • The implementation of the Project. . ^ . : 

Arrangements, by. the .Project Director for th^ s^tes, supplies, 
personnel and consultants. 

\, 

• The production of the >final .version of the Evaluation Plan. 

the remaining six objectives in the Management Co^^onenf were 
" attdioed by the end of the yeatr. T\j^e objectives j:alle^^or: ^ ^ 

- / ^ H ^ 

' ll^e submission of quarterly reports to the State Department 

of Education. ' ^ , ^ 

\ 

« J The conducting of staff jjjjpeservice and Inservice. 

1/ . ' 

• The monitoring of tbe program- on ^ monthly basis. 

• The implementation of the Evaluation Plan. 



A program to inform parents of the programl 
Maintaining contact with parents. 



Speciiic Project attainment^ are detailed in the following portion 
of Table XII. . " ' 
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NOTE I 

Relative to contacts made by the Home-School Coordinators during 
the year, the Project objective states that 80% of the parents of Title 
I children will receive at least one home visit and at least 50% will 
be contacted^ secohd time. Summary figures for the number of home con- 

f 

tacts made by the Home-School Coordinator at each schrool are presented 
in Tables Xllla and Xlllb. 

In Table Xllla, the number of home visits (as well as^ the frequency 
of those visits) is displayed. In Table Xlllb, the percentage of parents 
receiving the minimal number of visits is displayed, 

TABLE Xllla ^ 
Nlimber of Parent* Contacts by Home-School Coordinators 



School 



Denali 
F^irview 
Mt. View 
North Star 

TOTALS 



Number of 
Table I 
Students 




1 


2 


3 or 
more 


156 


17 


54 


32 ' 


53 


102 


20 


10 


17 


55 


151 


21 « 


36 


31 


63 


11.3 


2 


18 


25 


68 












522 


60 


118 


105 


239 



TABLE Xlllb 



Parentis Receiving Parents Receiving 
School One Contact Two or More 

Denali' 89% 54% 

Fairview ' 80% / , 71% 

Mountain View 86% ' 62% 

North Star 98% . ' 82% 

The foregoing figures indicate that 89% of the District's 
ytle I families were visited at least once and at least 66% 
were contacted two or more times- 



• 



% 93 



Staff , and Materials Development Compgnent Overview 

There are ten objectives in the Project's Development 

Component. These objectives include five staff development 

objectives and five materials development objectives StaLf 

objectives 2, 3, 4a, and 5 have been attained, 

a. Staff Development Objectives 

Counting those meetings reported in the last 
Anchorage Report, the «t>aff has scheduled (and held) 
a total ot 18 training sessions, i«eX^ eight each at 
Fairview, Mountain View, 'and oDenali:. The attendance 
at the three spring sessions averaged 1007., which 

9 * 

exceeds the gri.teria level (90%). The average attendance 
for the year was 937.. • ' • ^ ^ ' 

A review of the Vrolect records, "^^s ^well as inter- 
views with staff teacfters arrd' a Quality Assurance 
Specialvst*'(QAS) indicates 'that the T^o J ec t materi al s 
and procedu'i^es *are : 

(1) ^ Being used' on a regular basis in e^ch of the . 

Project classrooms (Objective 2. a. 4. a) 

(2) Being used in a mariner which, is rated as 
'^satisfactory** or better (Objective 2«a.4.b) 

(3) Being monitored on a periodic basis by the Project 
Quality Assurance Specialist (Objective 2. a. 5.) 

Documentation of the staff activities within this 

component appears to be excellent. The Project Management 

Team" current ly reviews the teacher ratings on a periodic 

basi s. 

* . -) 

Specific Project attainments on lach staff development 

. ^ / . 

objective are detailed in the following portion of Table XII. 
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Staf[ and Materials Dev&Iop--ient: Component: Ovegview (continued) 
^'b. Materials Develo{3ment Objectives 

Of the five* materials development^objectives, two were 
completed earlier,* two vvrere i^ated as partially attained and 
one was rated a suspended judgement because the methods..^ed 
differed from those specified in' the Evaluation Plan. The 
•most significant work accomplished in this compo1:ient since the 
last report is the testing on a sample basis of the concept 
test which has been tried with 120 children- The results are 
presented in Figures 2-17 and Table V, pages 29-37. 
^ • The following pages summarize Vhe Project's accomplishments 

in the area of Materials - Development, 
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3. Instructional Component Overview ^ 

The Program's Instructional component contaJjis eight objectives. 
Four of Dhese objectives represent expected student performances while the 
four represent supporting instructional activities. 



' UU L U 



Student performances in the concept areas, (1) food marketing, 
(2) transportation, (3) communJLcations , and (4) career awareness, did 
not reach the criterion levels. ^ 

Project students came closer to attaining the Language lObjectij^es with 
53 percent of the Kindergarten students and '64% of the 'First Grade students 
meeting the criterion of a thirty point gain on the SWCEL Oral Language 
Test. 

* 

On the average, project students gained between ^eight and ten months 
(grade level equivalent) in reading. In general, however, the performance 
of the present project students did not exceed that of the baseline stu- 
dents. 

0 

» 

A review of project activities and records indicated that all instruc-^^^ 

i 

tional process objectives had been attained. 

A summary of the instructional poraesses provided through the ^itle 
I Program is presented in Table XIV, 
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Table XIV 

Supplementary Instruction Provided by .Title I 



SWCEL Oral 
Language 



UYA* 



< ^ Home- 
Oral School 
Reading ^ Language . Coordi- Teacher 
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No 


Yes 


Yes 


Yes 
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Yes 


View 


Yes 


Yes 


Yes 


Yes 


North 




1 
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No 


1 Yes ' 

1 
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No 
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* University Year- of Action (UYA) 

Includes field trips and concept' lessons 
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NOTE 2: ^ 

leather j/dgiaents were made regarding **the placement (or non^laciment) \ 
of students iAto the SWCEL program. These judgmertts were Subsequently 
compared wi^h student performance on the SWCEL test. According to the test 
manual, those students scoring between 130 points should be placed into 
the program. In the present instance, an error in teacher judgment was 
recorded if (a) teachers failed t^ nominate pupils for the program who 

0 

scored bAow the critical value (overestimation) , or (b) teachers nominated 
pupils for the program who scored above the critijcal value (underestimation) . 

As evidenced in Table XV, most teachers tended to overestimate their 
stU4ents. Two teachers consistently under estimated their students, but this 
apparently occurred as a result of a "blanket'* nomination of all students 
in the cl^ss for the SWCEL program. 

The cori^lation between teacher judgments and ^udent'test scores is 
indicated by tK^ final column of Table XV(^b)- The correlations (r-biserial) 
range from ,08 to^A . These correlations indicate a need for improved 
judgments on the part, of some teachers. 
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T^^LE XV 

T€A£;HEF» PLACEMENT VS. SWCEL TEST SCORES 
Fall, 1973 



GRADE 


TOTAL N 


• 

TOTAL 


ERROR 


% OF ERRORS 
U V tiK** 
ESTIMATED 


% OF ERRORS 
IFMnFR- 

U Li L/C* 1\ 

ESTIMATED 


r 

b 


Kindergarten 


24 


. 7 


(29%) 


25% 


4% 


.6.1 


- 


47 


26 * 


(5^%) 


51% 


4% 


.64 




42' 


14 ■ 


(33%) 


31% 


2% 


.59 




43 


22. . 


(51%) 


46% 


57. 


.57 
















Grade 1 


29 


9 


(31%) 


24% 


77. 


.86 




31 


13 


(42%) 


36% 


6% 


.08 




20 


3 


(15%) 


10% 


5% 


.63 




18 


6 


(33%) 


22% 


11% 


.50 




23 


3 . 


(13%) 


' 9% 


4% 

i 


.86 




21 


^ 9 


(4 3%) 




A3Z 






20 


12 


(60%) 




60% 






24 


3 


(12%) 


8% 


4% 


.94 



Average percent .of errors per 'teacher: 39% 
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SUMMARY DISCUSSION 



As evidenced by the performance of Project children and the 
comments of staff members (see Section IVC), there are a number of areas 
where changes to the present program would be appropriate. In the paragraphi 
which follow, a number of such areas are discussed, ^ 

1. A change in the Monitoring Forms 

The Quality Assurances Specialist (QAS) staff appear to 
be reluctant to indicate an "unsatisfactory" performance by 
any teacher. The reluctance is understandable, but the moni- 
toring did oc'cur and it does riot accurately reflect the areas 
where the QAS staff worked out changes with the Project teachers 
^ and aides. 

I** ^ 

Recommendation. 
— — $ 

'The Program Evaluators recommend that the monitoring forms: 

(a) be simplified into a chejcklist where possible. 

(b) be constructed to assure a certain anon)nnity for 
teachers. The forms should contain information of 
an "alerting" nature rather than pinpointing the 
specific actions of a specific teacher in documented 
and possibly libelous form. 
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Addi cionally, it is recommended that the QAS staff be 
detached from their present school positions. This action 
would perrlt their use as technical resource persons to all 
Title I staff members, regardless of school location. 

/ 

Developnent of the Project Model 

Having the Project staff jointly develop the "Project 
Uodely'' (see Chapter III, pages 7-10), apparently serves to 
increase the staff's understanding of the Program. 

Recommendation 

It is recoKimended that the Project staff collectively 

^ 

engage in the writing and/or review of the Project Model 
before school begins in the fall, j 

Gommutfi cat ions Network 

A more effective communications network is needed 
within the Project. 
Recommendati on 

It is recommended 'that monthly meetings be scheduled 
on a regulai; basis between the Project Coordinator, the QAS, 
and: . » 

(a) Principals 

i 

(b) Home-School Coordinator * 

(c) Reading Specialists 

(d) Oral Language Specialists 

13^ 



4. Teacher Rati ngs and Student Performance 

J 

During the year, one of the program's screening procedures 
vas studied, i.e. the relationship between teacher ratings 
and student performance on the SWCEL Test of Oral English ^ 
Production. Essentially the congruence between teacher 
ratings and student performance varied betwe^^ 40 and 88 per 
cent with an average of 61 per cent. ^ 
Recommendation 

Because teacher ratings provide the basis for the 
initial selection of Project students for the SWCEL Oral 
Language Program, it is important that errors in these 
ratings be minimiz^ed. Accordingly it is recommended that the 
individuals who administer the Test of Oral English Production 
either make the initial assessment or assist the teachers in 
maki ng these ratings. 

5. Pre«Post Concept Testing 

According to the Project Evaluation Plan, Project students 
w^re to be administered the Project's four concept test^ on 
^ a pre-^st basis. This implies that one, all-inclusive pre- 

test vould be givpn in the fall of the year and that a sub- .« , 
sequent post-test would be administered during the spring". 
As noted by the Oral Language Specialists, this interval may 
be too great to.^ermit accurate assessment of student 
attainments on specific concepts. 
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Rcconrendation 

Tn.c Oral Language Specialists have suggested that 
pre and post testing for each concept unit be administered 
at the time the unit|is taught. In addition, it is recommended 
that a delayed follow-up test be administered on a sampling basis. 

Process Evaluation Procedures 

The Southwest Cooperative Educational Laboratory 
providesvseveral forms of technical assistance to the . 
Proj%t. One o^ these is a process evaluation procedure* 
'^According to the description of this procedure, each 
participating teacher is to administer and score a series 
of '*end of unit" tests during the year. Student response, 
protocols are then forwarded to SWCEL for analysis. 

As described in the Quality Assurance Model for Process 
Evaluation, the analysis consists of determining each 
classes average, determining the average of all classes, 
and conversion of all of the averages to standardized 
T-rSCores. The T-scores for all teachers are then plotted 
on graph paper and r^urne.d to the Project. (See Figure 19) • 

During the current year there were three flaws to this 
procedure : 

(a) The class means w^ie apparently not returned in time 
to be of u^e to the teachers. The analysis from 
tests administered in October were re^'urned in May. 



Tlie interpretations based upon the foregoing 
analysis are subject to error. To bo specific, 
a change in the relative standing of a class 
from one tost to the next does not necessarily 
yield the conclusion that **the teachers who show 
an increase in their T-scores are theoretically 
using behaviors that allow students to learn 
at a greater rate than anticipated,'* Nor does it 
necessarily follow that the declining graph line 
would indicate teacher whose students are not 
performing at their expected level. . • 

In the first place, the plot^s of T-scores 
primarily indicate the standing o^ a group of students 
relative to other groups of studeri:s. In' no way 
is an '^anticipated** or "expected level" of performance 
defined through the use of T-scores. Unless tl^e 
"expected level" of performance is defined as the 
overall mean of all groups, such a definition is 
tantam'ount to saying that the "expected level" 
of performance of the children in the Project is 
the group mean. It is not clear that such an expect- 
ation is either reasonable or desirable. 

A more serious drawback to the proffered 
interpretations, however, is the variety of plausable 
alternative explanations for "changes" i n the performance 
of stud^ntifi. The following variables are presented 
for purposes of illustration: 

(1) Measurement error which is inherent in 
every test. 

(2) Changes in student enrollment in the r 



(3) Changes in composition of the group 
being tested due to daily absen(^es. 

(4) .Changes in the difficulty levels of both 
tests and content area. 

These variables then mitigate against a clear interpre 
tation of changes being ascribed to teacher performances. 



A third drawback revolves about the use of relative 
standards in making judgments about a teacher's 
activities. Consid'er the instance in which the 
performance level of chi^ldren declines from test 
to test. A teacher who is fairly consistent in 
her approach would appear to "rise" as though she 



program. 
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wtrre '^improving, Conversely, in the light of a 
generally rising level of performance, the fairly 
consistent teacher is seen as ^'getting worse." 
In both of these instances the literal interpretation 
of i^he graph yields erroneous conclusions. 



Recorroendation 

For these reasons, it is irecommended that the present 
graphic "feedback" mechanism not be employed in its present 
form during the coming year. 

7. Positiipn of Project Coordinator 

The pQsition of Pro j ec t Coordi nator (.75 FTE) has been 
very abl} handled during the present year. However, the 
position requires a full-time commitment to the program. 
Rec ommendat i on 

In view of current needs and in anticipation of an 
expanded program next year, it is recommended that this 
positipn be maH^^ a full-time position for the coming year. 

v ^ 

33 _ Program Instructional Procedures 
Concept Areas: 

Based upon the data which have been compiled and reported 
in Chapter IV of the present report, it is apparent that stu- 
dent performance in the four concept areas was less than anti- 
cipated. For example, only in the area of Food Marketing (for 
third grade students) did the performance meet the eT^tablished 
criteria, 

"J 3/ 



j At least two .factors may have contributed to this apparent 

levell of performance. First, although 41 lesson plans dealing with 
the various concepts were developed and distributed, it is not clear 
that tne lesson plans were enacted in a uniform manner within the 
various project classrooms. Second, it is apparent that the project^ 
current set of lesson plans have not been organized in a sequential 
- or hierarchical manner. Thus, it may be the case- that lessons 
(generally centering about field trip activities) were provided where 
little instruction was needed.. Considerations such as these have 
led to the following recommendations. 

Reconmendat ions 

It is recoramenAd that the instructional lesson plans be upgraded. 
To that end, the Project Coordinator has been provided with intro- 
duction (including one site visit) to projects which are engaged pri- 
marily in the production of self-contained learning packets organized 
along hierarchical lines. 

Further, it is recommended that the staff review the present 
scope and sequence of concepts within each concept area. Delineation 
of concepts which are appropriate to younger (vs. older) children is 
recomiended. The item analysis which was provided in the present 
report may serve as a starting point fot this latter activity. 

8b . Program Instructional Procedures 
S:;CEL Oral Language Program: 

.\lthough the project objectives of a 30 point gain for 80% of 
the students was not achieved, the SV7CEL Oral Language Program gen- 
erally performed in accordance with the expectations of program. staff 
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menbers^uring che past year. -The gains made by program students 
wer.^ consVderable. One difficulty, however, was that of misclassi- 
fication. Errors in classification led to the selection of children 
into the SWCEL Oral Language Progrmn who did not necessarily need 
the program. The errors in part wete brought a^out by the delayed 
return of the SUCEL Oral Language Test results. Additionally, 
based upon an earlier analysis, it was determined that teacher judg- 
ments (which actua-lly served as the selection procedure duriag the 
'past year) were somewhat unreliable. 

Recommendations 

It is recommended thdt the agency which provides scoring 
vices be obligated to provide faster "turn-around" of the 
(i.e., selection data). 

«> o 

It is further recommended that a portion of the, pre service 

program be devoted to the SWCEL t.est so that teacher ju^lgments are 
better aligned with SWCEL test results. 

8c. ^ Program Instructional Procedures 

Reading; / v_ 

' The major findings relating to the current year.' s- reading program 
were both positive and negative. On the one hand, program children 
across all grade levels averaged between eight and ten months gain 
in reading achievement. On the other hand, since these children began 
the year somewhat behind in their reading, performances, gains of such 
magnitude were generally not sufficient to keep' them from slipping 
further. -behind. In {^articular, by the end of the first grade, the 

13,1 



project children. 00 fei^e average were two months behind; by the end 
of the seccmd grade, project children were, on the average, four ^ 
months behind; and by the end of the third grade, project^ children 
were, on the average, ten months behind. Of course, it must be 
recognized that the project *^j|^jor hypothesis dealing with the 
establishment of basic concepts, and the enhancement of language 
skills as ptecurs^^s to the acquisition of reading skills, has not 
yet^^rebeived a full test, since the other language and experience 
aspects of the program are presumed to have a cumulative effect. 
On the other hand. Title I reading instruction apparently varies 
considerably *fro\n child to child as well as from school to school, 

Recommendat ion 

Because of the evident success which the SWCEL program has had 
during the past year, it is recommended that the Program reading 
staff consider the establishment of a uniform array' of skills and 
a concentrated and uniform approach to the attainment of those 
ski/lTs. 



END OF REPORT 



140 



ERIC 



133 



( 



EXHIBITS 
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Materials Development Objective 5 



Student_ 
Grade 



Teacher 

Tester^ 

Date 



TITLE I, CONCEPT ORAL MASTERY TEST 
COMMUNICATION 

1. What are three ways we ^ Any 3 legitimate answers (i.e., 

get the news? newspaper, radio, television). 

. Prompt *'Can you name any other..? 



2. Can you name a section 
of the newspaper? 

another? 



3. How are radio and tele- 
. vision alike? 



Any .2 legitimate answers (i.e. , 
editorial, comics, ads). Prompt 
"Can you name any other...?" 



Voices, heard over long dis- 
tances, etc. 



( 



4. How are radio and tele- 
vision different? 



5. Why isra neswpaper im- 
portant? 



Picture on television 



Arty legitimate answer 



6. Can you name something 
that helps us learrr in 
the classroom? 

another? 
anymore? 

7. How can you tell some- 
thing to someone far away? 



Allow reference dealing, with 
people, but prompt to get 3 ' 
other responses. 



\Any legitimate answer (mail, 
telephone, telegram, etc.) 



8. How did people send mes- 
sages before the tele- 
phone was invented? 

9. How can you tell if your 
teacher is happy without 
her telling you? 



Any legitimate answer (smoke 
signals,' telegraph, runners, 
mait ,> etc. ) 

Any legitimate answer (laughing, 
smiling, etc.) 



10. Can you name a way you 
can express your feelings? 
another? 
anymore? 
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Any 3 legitimate answers^ 

Prompt "Can you name any others?" 



waceriais Development Objective 5 



Student^ 
Grade 



Teacher_ 
Tester_ 
Date 



TITLE I, CONCEPT ORAL MASTERY TEST 
TRANSPORTATION 



. Can you name' a kind of 
boat? . 

another? 
anymore? 



2. Can you name 2 kinds of 
trucks? 



Any 3 legitimate answers, 
Prompt "Can you name, any 
other....?" 



Any 2 legitimate answers. 
Prompt "Can you name any 
other? 



3. What do you call a boat 
which is used to push or 
pul 1 a large ship? 



Tugboat 



4. What is the fastest way to 
travel from one city to 
another? 



Airplane or jet 



5. What is a kayak? 



A skin boat originally used' 
by natives or similar answer. 



6. What are the two ways 
you can travel through 
the snow? 



Any 2 legitimate answers. 
Prompt "Can you name any 
other? 



7. Can you name a way you Any 3 legitimate answers. 

can get to school? r— Prompt "Can you name any other?" 

another? 
anymore? 

3. What are 2 ways to Any 2 legitimate answers (i.e,, 

carry big boxes of ap- trucks). Prompt "Can you , 

pies from one city to name any other ..?" 

anotH^r? 

9. What are 3 ways of tra- • Any 3 legitimate answers. 

veling through the air? Prompt "Can you name any 

other...?" 



10. What is the difference Difference concerning wings, 

• between a jet and a motor, shpae, etc. 

helicopter? 143 



Material Development "^Objective 5 



11 2 



s\udent 
Grade 



1. What are 3 anfmals we get 
food from? i 



Teacher 
Tester 

7 

Date 



TITLE I, CONCEPT ORAL MASTERY TEST 
FOOD MARKETING 



Any 3 legitimate answers. 
Prompt "Can you name any 
other. 



2. What is bread made from? 



Wheat, flour, yeast, etc., 
acceptable. 



3. What are 2 foods we get 
from the ocean? 



Fish, shellfish, seaweed, etc, 
If answer is 2 types of fish 
(i.e., cod & salmon), prompt 
"Can you tell me a different 
kind of food?" 



4. Can you name an animal that 
lives on a farm? 

another? 
anymore? 

I 

5. Where does thi grocery 
store get the food it 
sells? 



6. Can you name a vegetable? 

another? 
anymore? 

7. Name a dairy food you 
should have every day. 



Any 3 legitiniate answers. 
Prompt "Cafi you name any 
other...?" 



Primary sotfrce: farmers, fish- 
erman, or similar. One prompt 
permitted (i.e., where does 
the truck get it from?) 

Any 3 legitimate aasweri3: 
Prompt "Can you name any 
other? 



Milk 



8. What are 3 fruits that 
come from trees? 



Any 3 legitimate answers. 
Prompt "Can you name any 
other..?" 



[ 9. Can you name a dairy 
food? 

another? 

^ anymore?' 

10. Can you name one of the 
Basic Food Groups? 

another? 
anymore? 



Any 3 legitimate answers. 
Prompt "Can you name any 
other...?" 



Any 3 legitimate answers. One 
question and a maximum of 3 
prompts. Break off after first 
wrong response. 



Materials Development' Ob jective 5 



Student_ 
Gnde 



Teacher^ 
Tester_ 
Date 



TITLE I , CONCEPT ORAL MASTERY TEST 
CAREER INPOPMATIQN 



1. What does the Coast* Guard (or 
Navy, Air Force, or Army) do? 

2. What is one job you could have 
on a ship? 

3. What are 2 things you could do 

if you worked at ? 

(grocery store or gas station) 

4. IVhat are 2 jobs you could have 
if you worked on a railroad? 



5. What are 3 jobs you could have 
if you worked at the airport? 



6. What are 3 things a fanner 
does? 



Any legitimate answer. 
Any legitimate answer. 



Any 2 legitimate answers • 
Prcnqjt "Can you name any 
others ?" 

Any 2 legitimate answers. 
Prompt "Can you name any 
others ?" 

Mechanic, flight controller, 
ticket agent, etc. Must be 
directly related. 



Any 3 legitimate answers. 
Pronpt "Can you name any 
others ?" 



?• What does a butcher do? 



Any legitimate answer. 



Can you name a job you cou^d 
have if you worked for the 
telephone coinpany? 
Another? 

What are 2 things a mailman 
does ? 



10.' What are 2 things you could 
do if you worked for a news- 
paper? 



Any 2 legitimate answers. 

^ Pronq)t '*Can you name any 
" others 
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Picks mail, delivers mail, 
etc. Pron5)t "Can you name 
any others ?" 

Any 2 legitimate ansvers. 
Prompt "Can you name any 
others ?" 
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